The present invention relates to event detection, and more particularly to a computerized system and method for detecting events based on audit data received from one or more audit sources. Even more particularly, the present invention relates to the detection of intrusion or misuse event detection in a distributed computer network environment based on audit data generated at one or more audit sources, such as operating systems running at remote computer installations.
Misuse detection is the process of detecting and reporting uses of systems that would be deemed inappropriate or unauthorized if known to the responsible parties. Even though designers, owners, and administrators of such systems usually try to prevent misuses, the complexity of modern system environments and the difficulty of preventing authorized users from abusing their privileges makes it virtually impossible to anticipate and prevent all possible security problems. To date, however, there is no known system or method for effectively and independently detecting and reporting misuses and facilitating their subsequent investigation.
This misuse detection and reporting research has followed two basic approaches: anomaly detection systems and expert systems, with the overwhelming emphasis on anomaly detection.
Anomaly detection looks for statistically anomalous behavior. It assumes that intrusions and other security problems are rare and that they appear unusual when compared to other user behavior. D. Denning, xe2x80x9cAn Intrusion Detection Model,xe2x80x9d Proc 1986 IEEE Symp. Security and Privacy, (April 1986) provides an anomaly detection model (hereinafter the xe2x80x9cDenning Modelxe2x80x9d) for detecting intrusions into computer systems. The Denning Model uses statistical profiles for user, dataset, and program usage to detect xe2x80x9cexceptionalxe2x80x9d use of the system.
There are variations of the Denning Model of anomaly detection models and different applications of these models.
Expert systems (also known as rule-based systems or production systems) also have had some use in misuse detection, generally as a layer on top of anomaly detection systems for interpreting reports of anomalous behavior.
S. Snapp, et al., xe2x80x9cDIDS (Distributed Intrusion Detection System)xe2x80x9d Proc. 14th Nat""l Computer Security Conf., Washington, D.C. (October 1991) describes one example of an expert system signature analysis model that detects misuse by looking for one specific event within a specific system context.
In general, a computer event detection system is designed to protect a computer installation, with which it is associated, against abnormal computer actions of users (i.e., both insiders who are entitled to use the computer installation normally and outsiders who have intruded into the computer installation), whenever such actions are likely to give rise directly or indirectly to a breach of confidentiality, of integrity, and/or of availability of data and services from the computer installation.
The direct financial impact of computer misuse is very high and increasing. The National Institute of Justice (NIJ), for example, estimates the cost of computer misuse in the U.S. during 1993 to range from $500 million to $5 billion dollars. In addition, NIJ estimates that $2.1 billion was lost in the same period from telephone service fraud through illegally manipulating telephone company computer systems. In fact, virtually every sector of modern commerce and government, from banking to national defense, depends on the security of processing systems on which the sectors rely. As an increasing number of organizations connect their internal networks to outside public networks (e.g. the Internet, xe2x80x9cNational Information Infrastructurexe2x80x9d, etc.), the potential importance of misuse increases. This is because vulnerability increases with increased exposure.
Processing system misuse detection and reporting research has been funded by U.S. government agencies who have concerns for the confidentiality of their computer systems. Researchers have generally been associated with large research organizations or national laboratories. These institutions have required detailed knowledge of technical computer security, known threats and vulnerabilities, protection mechanisms, standard operational procedures, communications protocols, details of various systems"" audit trails, and legal investigation of computer crimes.
A computer event detection system, as mentioned above, is designed to detect abnormal computer actions constituting intrusions of insiders and a fortiori of intruding outsiders, and also to detect the people involved in such intrusions or suspected of being so involved.
Numerous present-day computer installations, whether they have centralized processor units or are they organized in networks of interconnecting geographically distributed processor units, have various access points for serving their users. The number of such points and the ease with which they are often accessible, as well as the requirements necessary for running such computer installations, have the drawback of facilitating attempts at intrusion by people who are not authorized users, and attempts by users of any kind, whether acting alone or in concert, to perform potentially harmful computer operations.
It is known that detecting intrusions into a computer installation and identifying the users performing illegal actions can be attempted by an approach that is statistical or neural, as mentioned above, or based on an expert system, as also mentioned above.
U.S. Pat. No. 5,557,742 (Smaha et al.), incorporated herein by reference, describes a method and system for detecting intrusion and misuse of data processing systems. The system uses processing system inputs, which include processing system audit trail records, system log file data, and system security state data information to detect and report processing system intrusions and misuses. A misuse selection mechanism allows the detection system to analyze the process inputs for a selected subset of misuses. The processing system inputs are then converted into states that are compared, through the misuse engine, to a predefined set of states and transitions until a selected misuse is detected. Once a misuse has been detected, an output mechanism generates a signal for use by a notification and storage mechanism. The detection system then generates a text-based output report for a user to view or store.
A number of false positives is minimized by creating signatures from undesirable activities including known attack outcomes, known system vulnerabilities and known attack procedures. Misuse is only reported upon a direct match to a known misuse signature; the probability of falsely reporting a misuse is reduced over the previous anomaly detection mechanisms. The signatures are generated by a programmer and are loadable at program initiation. System programmers are capable of creating their own misuse signatures from their particular known attack procedures, attack outcomes, and known system vulnerabilities. The misuse signatures are deterministic, unlike expert systems. The system does not use statistical analysis.
U.S. Pat. No. 5,621,889 (Lermuzeaux, et al.), incorporated herein by reference, describes a facility for detecting intrusions and possibly suspect users, by making use of streams of surveillance data relating to the operation of a computer installation and in particular to the actions of users of the installation, wherein the actions take place at installation level. The detection facility involves modeling a target as constituted by the computer installation and its users and also their respective behaviors by making use of previously acquired knowledge and rules and by making use of a symbolic representation using a semantic network; comparing the modeled behavior of the installation and its users with normal behavior expected for the same conditions as modeled by the behavior rules and security rules contained in a knowledge base, and inferring therefrom either an anomaly object in the event of at least one of the behavior rules being violated, or an intrusion object or an intrusion hypothesis object in the event of at least one of the security rules being violated; interpreting the anomalies observed by implementing the previously acquired rules and knowledge so as to generate, reinforce, or confirm corresponding intrusion hypotheses; correlating and interpreting intrusion hypotheses and intrusions observed by implementing the previously acquired rules and knowledge in order to link together various intrusion hypotheses and/or intrusions, and to infer new intrusion hypotheses and/or intrusions therefrom; and co-operating with the various other steps above to indicate the various data items that the various other steps above produce relating to behavior, anomalies, intrusion hypotheses, and intrusions.
The present invention advantageously addresses a need for improvements in event detection systems.
The present invention advantageously improves upon heretofore known event detection systems by providing an event detection system, and more particularly a computerized event detection system that detects events based on a plurality of detection models in a highly secure distributed processing environment based on audit data received from one or more audit sources.
The present invention in one embodiment, can be characterized as a system for event detection. Such system employs a collector that collects raw audit data made up of raw audit data records at an audit source; a database; an inserter at a downstream processing location that inserts Virtual Records into the database, including both a first type of Virtual Record generated in response to a raw audit data record, and a second type of Virtual Record generated in response to a detected audit event, the inserter; a parser, coupled to the collector, that converts raw audit data records in the raw audit data into Virtual Records; a detector that detects audit events in response to the Virtual Records generated by the parser, and generates the second type of Virtual Record in the event an audit event is detected.
In a variation of the system of the one embodiment, the system further employs a sender coupled to at least one of the collectors, the parser, and the detector, at the audit source, for sending output data from at least one of the collectors, the parser, and the detector to a downstream process location.
In an additional variation of the system of the one embodiment, the sender includes an encryptor for encrypting output data from at least one of the collectors, the parser, and the detector before the output data is sent to the downstream process location.
In a further variation of the system of the one embodiment, the detector is coupled to an output of the parser, and the detector detects audit events in response to the Virtual Records generated by the parser and generates Virtual Records of the second type in response thereto.
In yet another further variation of the system of the one embodiment, an input of the detector is coupled to an output of the database, and the detector detects audit events in response to the Virtual Records in the database, and generates Virtual Records of the second type in response thereto. Also, an output of the detector may be coupled to the inserter, so that Virtual Records of the second type generated by the detector are inserted into the database by the inserter.
In another further variation, an output of said detector is coupled to the inserter, and Virtual Records of the second type generated by the detector are inserted into the database by the inserter.
In an additional variation, the event detection system also includes a user interface coupled to the database. The user interface filters Virtual Records in the database based on a filter criteria.
In variations, the detector may perform rule-based analysis of the Virtual Records generated by the parser, and/or may perform statistical analysis of the Virtual Records generated by the parser. When performing statistical analysis, the detector determines a mean and a standard deviation of a parameter in a set of the Virtual Records, and detects in a subsequent Virtual Record whether the parameter is more than a predetermined multiple of the standard deviation away from the mean.
In another embodiment, the present invention can be characterized as a method of event detection. The method includes steps of collecting, using a collector at an audit source, raw audit data comprising raw audit data records and communicating the raw audit data to the parser; parsing, using the parser, the raw audit data records by converting each raw audit data record into a Virtual Record of a first type and communicating the Virtual Records to a detector; detecting, using the detector, audit events in response to the Virtual Records generated by the parser, and generating a Virtual Record of a second type in response to each detected audit event and communicating the Virtual Records to an inserter; and inserting, using the inserter, Virtual Records of the first type and Virtual Records of the second type into a database.
In a variation of the method, output data from at least one of the collector, the parser and the detector may be encrypted before communicating the output to a downstream process location.
In further variations of the method, rule-based analysis and/or statistical analysis is performed on the Virtual Records. The statistical analysis may be include determining a mean and a standard deviation of a parameter in a set of Virtual Records; and determining whether in a subsequent Virtual Record the parameter is more than a predetermined multiple of the standard deviation away from the mean.